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Abstract 

In  support  of  Marine  Corps  Vision  and  Strategy  (MCV&S)  2025  Task  1  to  “Improve  small  unit 
leader  ability  to  assess,  decide,  and  act  in  a  more  decentralized  manner,”  the  USMC  Training  and 
Doctrine  Command  (TECOM)  created  the  Small  Unit  Decision  Making  (SUDM)  Initiative.  One 
objective  of  the  initiative  is  to  develop  a  SUDM  Assessment  Battery  to  understand  the 
development  of  decision  making  proficiency  of  infantry  small  unit  leaders.  The  purpose  of  this 
report  is  to  summarize  previous  work  leading  to  the  development  of  an  Initial  SUDM 
Assessment  Battery  and  to  highlight  the  development  of  two  customized  assessment  instruments 
within  the  battery.  The  battery  is  based  on  a  developmental  model  of  maneuver  squad  leaders 
and  on  a  multi-dimensional  approach  to  determining  decision  making  proficiency.  An  Initial 
SUDM  Assessment  Battery  was  developed  from  a  literature  review  of  available  instruments  and 
the  creation  of  three  custom  instruments.  Future  usability  testing  of  the  custom  instruments  will 
be  followed  by  a  pilot  study  of  the  initial  battery  and  finalization  of  the  battery  based  on  pilot 
study  findings. 


li 


CPG-0001 AE-21  Decl  2 


Contents 

Abstract . ii 

Figures . iv 

Tables . iv 

Executive  Summary . v 

Introduction . v 

Method  for  Designing  Custom  Instruments . v 

Findings . vii 

Utilization  of  Findings . vii 

Introduction . 1 

Background . 1 

Multi-Dimensional  Measurement  of  Decision  Making . 1 

Maneuver  Squad  Deader  Mastery  Model . 2 

Review  and  Selection  of  Available  Instruments . 4 

Usability  Testing  of  Off-the-Shelf  Instruments  and  BARS . 6 

Method  for  Developing  Custom  Instruments . 8 

The  Situational  Judgment  Test . 8 

Background . 8 

Methodology . 8 

The  Decision  Requirements  Interview . 10 

Background . 10 

Methodology . 11 

Results  and  Discussion . 13 

References . 15 

Appendix  A:  Operationalized  Definitions  and  Examples  of  SUDM  Competencies  and  CARS  A-l 


iii 


CPG-0001 AE-21  Decl  2 


Figures 

Figure  1.  Constructs  and  Initial  SUDM  Assessment  Battery  Instruments . 6 

Tables 

Table  1.  TECOM  Small  Unit  Decision  Making  Cognitive  Competencies  and  CARS . 2 

Table  2.  Nine  Key  Performance  Areas  for  Maneuver  Squad  Leaders . 3 


CPG-0001 AE-21  Decl  2 


Executive  Summary 

Introduction 

The  purpose  of  this  report  is  to  summarize  the  overall  development  of  an  Initial  Small  Unit 
Decision  Making  (SUDM)  Assessment  Battery  for  the  U.S.  Marine  Corps  (USMC)  and  to 
highlight  the  development  of  two  customized  assessment  instruments  within  the  battery. 

In  support  of  Marine  Corps  Vision  and  Strategy  (MCV&S)  2025  Task  1  to  “Improve  small  unit 
leader  ability  to  assess,  decide,  and  act  in  a  more  decentralized  manner,”  the  USMC  Training  and 
Doctrine  Command  (TECOM)  created  the  SUDM  Initiative.  One  objective  of  the  SUDM 
Initiative  is  to  develop  a  SUDM  Assessment  Battery  to  understand  the  development  of  decision 
making  proficiency  of  small  unit  leaders  (in  this  case,  Marines  who  are  currently  or  who  are 
expected  to  be  assigned  as  rifle  squad  leaders  or  weapons  section  leaders,  referenced  collectively 
throughout  this  document  as  maneuver  squad  leaders)  and  provide  insight  into  the  impact  of 
efforts  to  improve  proficiency.  To  address  this  requirement,  the  first  step  was  to  create  a 
developmental  model  for  maneuver  squad  leaders.  The  purpose  of  this  model  was  to  form  a 
foundation  for  understanding  the  general  progression  to  mastery  as  a  maneuver  squad  leader, 
including  the  progression  of  decision  making  proficiency.  Following  development  of  this  model, 
our  research  team  selected  a  number  of  relevant  instruments  from  the  literature  for  measuring 
constructs  supporting  decision  making  proficiency,  identified  gaps  in  instruments  available,  and 
developed  custom  instruments  to  form  the  Initial  SUDM  Assessment  Battery  for  future  pilot 
testing. 

The  SUDM  Assessment  Battery  is  based  on  a  multi-dimensional  approach  to  determining 
proficiency  in  decision  making.  Multi-dimensional  is  defined  as  addressing  multiple  constructs 
in  the  assessment  battery,  which  when  taken  together,  inform  our  understanding  of  an 
individual’s  decision  making  proficiency.  During  the  previous  work  conducted  by  our  research 
team  to  identify  existing  instruments  that  address  the  dimensions  or  constructs  that  are  part  of 
decision  making  proficiency,  we  determined  that  some  of  the  constructs  which  the  USMC  seeks 
to  measure  cannot  be  addressed  by  off-the-shelf  instruments.  Limitations  include  a  simple  lack  of 
instruments  for  the  construct,  inappropriateness  of  available  instruments  for  the  target  audience, 
and  a  predominance  of  self-report  instruments  found  in  the  literature  as  opposed  to  assessments 
that  require  performance-based  responses  which  we  concluded  were  warranted  for  some 
constructs. 

Method  for  Designing  Custom  Instruments 

This  portion  of  the  SUDM  Assessment  Battery  project  was  focused  on  custom  development  of 
two  performance-centered  assessment  instruments — a  Situational  Judgment  Test  (SJT)  and  a 
Decision  Requirements  Interview  (DRI).  The  SJT  scenario-based  items  were  derived  from 
critical  incident  data  obtained  during  mastery  model  interviews.  In-house  subject  matter  expert 
(SME)  review,  as  well  as  comparison  to  the  constructs  TECOM  seeks  to  measure,  resulted  in  an 
initial  version  of  25  items.  Future  usability  testing  will  support  finalization  of  the  items  and 
provide  the  foundation  for  determining  response  format  and  scoring  criteria  for  each  item. 
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Each  SJT  item  consists  of  a  brief  scenario  with  four  possible  courses  of  action.  The  scenarios  and 
courses  of  action  are  based  on  real  life  experiences  from  SMEs  interviewed  as  part  of  the 
mastery  model  development.  Interviewees  were  asked  to  describe  situations  where  they  (or 
someone  else)  had  to  face  a  difficult  problem  and  ways  a  squad  leader  at  different  levels  of 
proficiency  would  respond.  The  incidents  were  used  as  a  framework  for  developing  scenarios 
that  measure  the  cognitive  competencies  and  CARS.  Scenarios  were  further  edited  to  ensure  each 
item  was  of  similar  length  and  format  (McDaniel  &  Nguyen,  2001).  A  readability  analysis  was 
conducted  to  verify  that  the  language  would  not  add  an  unnecessary  cognitive  load  on  the 
examinee. 

To  more  comprehensively  address  the  lack  of  a  standardized  measure  for  addressing  the  overall 
decision  making  construct,  we  generated  the  DRI,  a  new  instrument  based  on  the  critical  factors 
for  good  decision  making.  The  factors  include  an  individual’s  recognition  and  interpretation  of 
relevant  information  and  indicators  from  the  environment,  understanding  of  the  complete  goal 
set  pertinent  to  the  situation,  and  typical  ways  of  apply  tactical  and  technical  skills  to  the 
situation  to  solve  the  tactical  dilemma  (e.g.,  Klein,  1998).  Our  primary  objective  in  developing 
the  DRI  was  to  produce  a  technique  that  is  not  reliant  on  SME  ratings,  allows  for  multiple 
acceptable  courses  of  action  (COAs),  enables  differentiation  between  acceptable  and  superior 
decision  making,  and  bases  ratings  on  an  individual’s  thought  process  and  rationale  rather  than 
the  outcomes  of  his  performance. 

The  DRI  was  conceptualized  as  an  interview  technique  consisting  of  a  tactical  scenario  with  a 
series  of  events  requiring  respondents  to  make  sense  of  the  events  and  decide  upon  a  COA  for 
each  event.  The  administration  protocol  includes  question  sets  to  elicit  the  respondents’ 
understanding  of  the  situation  and  considerations  for  decision  making,  and  a  Decision 
Requirements  Table  (DRT)  to  record  those  considerations  during  the  interview.  The  resultant 
DRT  would  then  be  scored  against  a  master  DRT ;  points  would  be  awarded  for  noticing  and 
correctly  interpreting  relevant  cues,  considering  important  factors,  identifying  goals  to  be 
achieved,  and  selecting  a  workable  COA. 

The  scenario  selected  was  originally  constructed  for  implementation  in  a  combined  live  and 
virtual  training  facility,  across  five  training  days  where  a  new  fragmentary  order  was  provided  at 
the  start  of  each  training  day.  We  selected  a  subset  of  the  scenario  events  to  include  in  the  DRI 
scenario,  and  then  reconstructed  it  into  a  paper-and-pencil  scenario  consisting  of  five  segments. 
Events  were  selected  with  the  goal  to  represent  a  range  of  operational  decision  types  and 
performance  areas  from  the  Maneuver  Squad  Leader  Mastery  Model.  Perceptual  cues  and 
indicators  originally  presented  via  live  or  virtual  scenario  events  were  re-written  for  introduction 
in  text  and  images.  A  mission  order  and  fragmentary  orders  were  created  to  ensure  respondents 
understood  the  mission,  the  operational  context,  friendly  and  enemy  situations,  unit  tasks,  and 
assets  available. 

The  scenario  consists  of  five  segments,  each  requiring  responses  from  the  participant.  The  DRT 
framework  for  collecting  responses  is  built  around  six  categories  of  decision  requirements  to  be 
understood  for  each  segment: 

•  The  dilemma,  decision  point,  problem,  or  issue  requiring  a  decision 
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•  Perceptual  cues  recognized  as  relevant  and  interpreted  to  support  effective  decision 
making 

•  Background  factors  recognized  as  relevant  and  considered  when  making  the  decision 

•  The  decision  or  course  of  action  made  by  the  participant  in  response  to  the  dilemma 

•  Common  errors  associated  with  making  the  decision 

•  Job  knowledge  applied  when  making  the  decision 

Findings 

A  version  of  the  Initial  SUDM  Assessment  Battery  (Ross,  Vogel- Walcutt,  &  Phillips,  2012)  was 
created  from  the  literature  review  and  the  custom  instruments  (BARS,  SJT,  and  DRI).  It  contains 
27  instruments  addressing  15  of  the  17  constructs  (excepting  anomaly  detection  and  change 
detection,  still  under  investigation).  This  version  is  subject  to  usability  testing  of  the  SJT  and  the 
DRI  to  be  carried  out  under  Option  1  of  the  contract.  Furthermore,  it  will  be  assessed  during  the 
conduct  of  a  pilot  study  and  modified  according  to  pilot  study  findings. 

Utilization  of  Findings 

Usability  testing  of  the  SJT  is  to  be  conducted  at  Camp  Lejeune,  North  Carolina,  with  six  SMEs 
to  determine  appropriateness,  face  validity,  and  clarity  of  the  items.  Each  participant  will  be 
provided  a  test  booklet  that  includes  SJT  items  on  separate  pages  with  instructions  and  space  for 
correcting  each  item.  After  the  usability  testing,  the  item  response  format  will  be  developed. 
Answers  scored  as  appropriate  will  be  based  on  SME  feedback  and  ranking  from  the  usability 
testing.  The  final  set  of  SJT  items  will  consist  of  SME  validated  scenarios  and  items  developed 
according  to  guidelines  in  the  scientific  literature. 

The  DRI  will  be  subjected  to  usability  testing  with  six  Sergeants  representing  the  target 
audience.  The  protocol  will  be  administered  to  judge  the  clarity  of  the  scenario  and  question  sets, 
the  protocol’s  ability  to  elicit  the  desired  elements  of  cognitive  performance  and  decision 
considerations,  and  the  researchers’  ability  to  construct  the  DRT  during  the  course  of  an 
interview.  Following  usability  testing,  the  scenario,  question  sets,  and  DRT  framework  will  be 
modified  as  needed  to  increase  the  utility  and  ease  of  use  of  the  DRI  as  a  performance-based 
measure  of  decision  making.  Once  this  interview  technique  is  modified  to  a  point  that  its 
administration  can  be  standardized  and  it  reliably  captures  decision  performance  data  as  desired, 
our  goal  will  be  to  modify  its  implementation  for  use  with  any  high-quality  tactical  scenario  and 
for  administration  by  non-researchers. 

Following  usability  testing  of  the  SJT  and  DRI  instruments  and  instrument  revision,  the  pilot 
study  will  commence.  The  study  will  consist  of  five  administrations  of  the  battery  as  well  as 
completion  of  subjective  surveys  by  participants  and  their  supervisors.  The  administrations  will 
be  conducted  prior  to  the  cohort  of  students  entering  their  Advanced  Infantry  Courses,  prior  to 
entry  into  the  new  Infantry  Small  Unit  Leader  Course  (ISULC),  at  the  conclusion  of  ISULC,  6 
months  after  assignment  to  a  billet,  and  12  months  after  assignment.  The  purpose  of  the  data 
collection  is  to  pilot  test  the  battery,  and  simultaneously,  battery  development  will  continue  as 
off-the-shelf  measures  are  reduced  in  number,  custom  instruments  are  refined,  and  additional 
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instruments  are  identified  or  developed  to  fill  gaps  or  better  address  constructs  than  do  initial 
instruments.  Some  changes  to  improve  efficiency  and  effectiveness  will  be  introduced  during  the 
pilot,  but  a  core  of  measures  will  be  maintained  across  the  pilot  administrations  to  insure 
consistent  repeated  measures  to  support  validity  of  insights  regarding  development  over  time. 
Some  changes  in  the  battery  resulting  from  analysis  of  the  pilot  data  will  be  introduced  only  after 
the  pilot  is  completed  and  the  SUDM  Assessment  Battery  is  finalized. 

Once  pilot  testing  is  underway,  we  expect  that  additional  gaps  may  emerge  in  the  battery.  For 
those  areas  where  the  currently  available  scales  are  too  cumbersome,  the  results  are  not  useful  for 
informing  training  practices,  or  where  no  scale  assesses  the  full  scope  of  a  construct,  new, 
military-targeted  scales  may  need  to  be  developed  and/or  measures  being  developed  under  other 
efforts  encompassing  the  constructs  must  be  identified  to  fill  the  gaps. 

We  recommend  that  versions  of  the  actual  battery  be  restricted  to  official  use  by  the  research 
community  only.  Knowledge  of  the  items  which  can  be  the  subject  of  discussion  among  potential 
instructors  and  student  participants  in  the  subsequent  pilot  testing  can  invalidate  the  findings. 
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Introduction 

The  purpose  of  this  report  is  to  summarize  the  overall  development  of  an  Initial  Small  Unit 
Decision  Making  (SUDM)  Assessment  Battery  for  the  U.S.  Marine  Corps  (USMC)  and  to 
highlight  the  development  of  two  customized  assessment  instruments  within  the  battery.  The 
SUDM  Assessment  Battery  is  based  on  a  multi-dimensional  approach  to  determining  proficiency 
in  decision  making  for  Marines  who  are  currently  or  who  are  expected  to  be  assigned  as  rifle 
squad  leaders  or  weapons  section  leaders,  referenced  collectively  throughout  this  document  as 
maneuver  squad  leaders.  Multi-dimensional  is  defined  as  addressing  multiple  constructs  in  the 
assessment  battery,  which  when  taken  together,  inform  our  understanding  of  an  individual’s 
decision  making  proficiency.  Previous  work  conducted  by  our  research  team  to  develop  the 
initial  version  of  the  assessment  battery  resulted  in  the  identification  of  a  number  of  existing 
instruments  that  address  the  dimensions  or  constructs  that  are  part  of  decision  making 
proficiency.  This  process  is  summarized  below.  Our  team  also  determined  that  some  of  the 
constructs  which  the  USMC  seeks  to  measure  cannot  be  addressed  by  off-the-shelf  instruments. 
Limitations  include  a  simple  lack  of  instruments  for  the  construct,  inappropriateness  of  available 
instruments  for  the  target  audience,  and  a  predominance  of  self-report  instruments  found  in  the 
literature  as  opposed  to  assessments  that  require  performance-based  responses  which  we 
concluded  were  warranted  for  some  constructs.  This  portion  of  the  project  was  focused  on 
custom  development  of  two  performance-centered  assessment  instruments — a  Situational 
Judgment  Test  (SJT)  and  a  Decision  Requirements  Interview  (DRI). 

Background 

In  support  of  Marine  Corps  Vision  and  Strategy  (MCV&S)  2025  Task  1  to  “Improve  small  unit 
leader  ability  to  assess,  decide,  and  act  in  a  more  decentralized  manner,”  the  USMC  Training  and 
Doctrine  Command  (TECOM)  created  the  SUDM  Initiative.  One  objective  of  the  SUDM 
Initiative  is  to  develop  a  SUDM  Assessment  Battery  to  understand  the  development  of  decision 
making  proficiency  of  small  unit  leaders  and  provide  insight  into  the  impact  of  efforts  to  improve 
proficiency.  To  address  this  requirement,  the  first  step  was  to  create  a  developmental  model  for 
maneuver  squad  leaders.  The  purpose  of  this  model  was  to  form  a  foundation  for  understanding 
the  general  progression  to  mastery  as  a  maneuver  squad  leader,  including  the  progression  of 
decision  making  proficiency.  Following  development  of  this  model,  our  research  team  selected  a 
number  of  relevant  instruments  from  the  literature  for  measuring  constructs  supporting  decision 
making  proficiency,  identified  gaps  in  instruments  available,  and  developed  custom  instruments 
to  form  the  Initial  SUDM  Assessment  Battery  for  future  pilot  testing. 

Multi-Dimensional  Measurement  of  Decision  Making 

Based  on  a  series  of  workshops  and  surveys  of  Marine  Corps  leaders,  Marine  Corps  subject 
matter  experts  (SMEs),  and  leading  researchers,  it  was  hypothesized  that  five  cognitive 
competencies  and  ten  cognitive  and  relational  skills  (CARS)  support  maneuver  squad  leader 
performance  and  decision  making  (U.  S.  Marine  Corps,  2011).  The  competencies  and  CARS  are 
depicted  in  Table  1  below.  The  SUDM  Assessment  Battery  is  designed  to  measure  each  of  the  15 
competencies  and  CARS;  overall  decision  making  proficiency;  and  level  of  mastery  for 
maneuver  squad  leaders.  The  SUDM  Assessment  Battery  addresses  these  17  constructs. 


1 


CPG-0001 AE-21  Decl  2 


Continued  work  as  the  battery  is  further  developed  may  reveal  relationships  among  these 
constructs  that  allow  for  the  consolidation  of  constructs  into  a  smaller  number  of  dimensions  that 
indicate  decision  making  proficiency. 


Table  1.  TECOM  Small  Unit  Decision  Making  Cognitive  Competencies  and  CARS 


Cognitive  Competencies 

Cognitive  and  Relational  Skills  (CARS) 

Sensemaking 

Cognitive  Flexibility 

Analytical  Reasoning 

Adaptability 

Resilience 

Perspective  Taking 

Problem  Solving 

Anomaly  Detection 

Ambiguity  Tolerance 

Metacognition 

Change  Detection 

Self-Awareness 

Attentional  Control 

Situational  Assessment 

Self-Regulation 

Maneuver  Squad  Leader  Mastery  Model 

To  support  improvements  in  assessment,  as  well  as  training  and  experience  for  maneuver  squad 
leaders  in  support  of  decentralized  operations  in  the  21st  century  hybrid  threat  environment,  we 
developed  a  Maneuver  Squad  Leader  Mastery  Model  (Ross,  Phillips,  Rivera,  Brown,  &  Smith, 
2012).  The  model  describes  the  developmental  path  to  expertise  for  rifle  squad  leaders  and 
weapons  section  leaders.  The  five-stage  descriptive  model  contains  key  performance  areas, 
performance  indicators  at  different  stages  of  development,  and  linkages  to  the  decision  making 
competencies  and  supporting  CARS  for  small  unit  leaders  that  were  previously  identified  by 
TECOM.  The  model  is  designed  to  (1)  provide  insights  into  how  individuals  progressively 
develop  into  high  performing  maneuver  squad  leaders,  and  (2)  provide  implications  for  what 
should  be  assessed  and  how  during  development.  These  insights  support  potential  actions  to 
improve  cognitive  readiness  with  individual,  unit,  and  organizational  enhancements. 

TECOM  requested  experienced  Infantry  NCOs  (Noncommissioned  Officers)  and  Officers  from 
five  organizations  to  participate  in  interviews  and  share  their  knowledge,  experiences,  and 
insights  into  the  key  performance  areas  for  maneuver  squad  leaders  and  the  path  to  development 
for  mastery  of  this  billet.  Interviews  serving  as  the  basis  for  this  report  were  conducted  at  School 
of  Infantry-East  (SOI-E),  School  of  Infantry-West  (SOI-W),  1st  Marine  Division,  2nd  Marine 
Division,  and  with  members  of  two  Reserve  battalions  preparing  for  deployment  as  a  Mobile 
Training  Team.  Participants  included  a  total  of  58  Marines.  Twenty  of  the  participants  were 
serving  in  Infantry  instructor  billets,  28  were  serving  in  Infantry  billets  in  the  operational  forces, 
and  ten  were  Reservists  preparing  to  be  instructors.  All  participants  contributing  to  the  Mastery 
Model  had  recent  combat  experience  except  for  one,  who  had  Marine  Expeditionary  Unit 
deployment  experience  to  various  countries. 

The  interviews  contained  over  874  references  to  behavioral  indicators  and  other  descriptors  of 
the  maneuver  squad  leader  as  development  progressed  through  five  levels  of  learning  and 
performance — novice,  advanced  beginner,  competent,  proficient,  and  expert  (Dreyfus,  & 
Dreyfus,  1986).  The  descriptors  were  separated  by  card  sort  into  the  nine  key  performance  areas 
by  stage  of  development  in  the  model.  The  nine  areas  and  definitions  are  shown  in  Table  2. 
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Profiles  of  performance  in  each  area,  for  each  stage  of  development,  were  developed.  Each 
performance  area  was  linked  to  the  competencies  and  CARS,  with  one  to  many  relationships 
found  across  performance  areas  (Ross  et  ah,  2012). 

Table  2.  Nine  Key  Performance  Areas  for  Maneuver  Squad  Leaders 


Key  Performance  Area  Definition 


Adaptability/Flexibility 

The  ability  to  fluidly  apply  knowledge  and  tactical  principles  across 
situations,  or  alter  one’s  plans,  actions,  or  decisions  when  the 
situation,  environment,  or  circumstance  has  changed,  while  still 
accomplishing  the  mission  or  intent. 

Administration 

The  coordination  and  supervision  of  people,  processes,  and 
equipment  in  conjunction  with  the  abilities  to  multitask  and 
delegate  assignments. 

Character,  Initiative,  and 
Command  Presence 

The  mental,  physical,  and  character  traits  of  an  effective  leader  who 
demonstrates  confidence,  sets  a  positive  example,  garners  respect 
and  trust  from  his  subordinates,  takes  full  responsibility  for  his  own 
actions,  and  accomplishes  tasks  and  goals  autonomously  within 
intent. 

Communication 

Effectively  obtaining,  relaying,  and  explaining  information  to 
subordinates,  superiors,  and  adjacent  squad  or  section  leaders  in 
order  to  direct  actions  or  maintain  shared  understanding. 

Job  Knowledge 

The  comprehension  of  procedures,  processes,  and  asset  capabilities 
required  to  effectively  perform  the  maneuver  squad  leader  role. 

Self-Control  and  Stress 
Management 

Managing  and  regulating  one’s  emotional  responses,  control,  and 
stability  in  order  to  prioritize  and  perform  effectively  within  high 
stress  contexts. 

Self-Development 

The  motivation  to  continuously  acquire  and  apply  new  knowledge, 
skills,  and  lessons  learned  to  current  role  requirements  and  future 
professional  development  goals,  as  a  result  of  an  attentiveness  to 
the  nature  of  one’s  self,  personal  strengths,  limitations,  and  work 
styles. 

Tactical  Skills/Tactical 
Thinking 

The  cognition  required  to  apply  tactical,  technical,  and  team 
knowledge  to  analyze  mission  requirements,  plan,  solve  tactical 
problems,  and  execute  the  mission  decisively,  within  the  big  picture 
and  Commander’s  intent. 

Train,  Mentor,  and  Develop 
Marines 

Continuously  caring  about  and  fostering  the  professional  and 
personal  development  of  subordinates,  by  teaching,  training, 
coaching,  building  trust,  assessing  skills  and  personalities,  and 
providing  guidance. 

Analysis  of  data  collected  for  the  Mastery  Model  interviews  informed  a  richer  understanding  of 
how  competencies  and  CARS  reveal  themselves  in  a  maneuver  squad  leader’s  actions. 
Definitions  of  these  constructs  previously  derived  from  the  research  literature  were  compared 
with  incidents  describing  actual  maneuver  squad  leader  experiences  and  decisions  from  the 
interviews.  The  competency  and  CARS  definitions  initially  derived  from  the  literature  were  then 
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operationalized  to  accurately  reflect  the  application  of  the  cognitive  constructs  to  maneuver 
squad  leader  performance  on  the  job.  Incident  examples  for  each  construct  illustrate  how 
maneuver  squad  leaders  apply  the  competencies  and  CARS  in  operational  or  garrison  contexts. 
These  operational  construct  definitions  will  improve  measurement  efforts  targeting  decision 
making  skills,  by  increasing  the  specificity  of  the  desired  measurement.  The  definitions  and 
examples  are  presented  in  Appendix  A. 

The  Maneuver  Squad  Leader  Mastery  Model  codifies  the  qualities  and  desired  performance  that 
must  be  considered  each  time  a  maneuver  squad  leader  is  selected,  each  time  an  educational 
program  is  instituted,  and  each  time  a  training  plan  or  initiative  is  developed.  It  describes  the 
developmental  progression  of  the  small  unit  leader,  to  inform  training  that  will  accelerate 
cognitive  readiness  and  approaches  that  will  accurately  measure  cognitive  performance.  The 
interviewees  responsible  for  selecting,  training,  and  developing  the  maneuver  squad  leader  have 
an  innate  understanding  of  the  person  they  are  looking  for  to  fill  that  billet  and  how  that  person 
can  reasonably  be  expected  to  develop  in  that  role.  Capturing  their  knowledge  and  experience  in 
the  model  allows  leaders  at  all  levels  and  the  training  and  education  community  access  to  their 
insights  with  a  comprehensive  and  documented  description  of  the  performance  demands  and 
requirements  for  success. 

The  Mastery  Model  also  forms  the  basis  from  which  to  measure  level  of  mastery  of  the 
maneuver  squad  leader  job.  Following  the  development  of  the  model,  a  Behaviorally  Anchored 
Rating  Scale  (BARS)  was  developed  to  address  the  construct  of  maneuver  squad  leader  mastery. 
A  BARS  instrument  pairs  observable  behaviors  with  numeric  ratings  of  performance 
(Muchinsky,  2003).  The  initial  BARS  for  maneuver  squad  leader  mastery  focuses  on  tactical 
thinking.  Of  the  874  references  to  performance  descriptors  in  the  data,  tactical  thinking  skills 
were  mentioned  with  the  highest  frequency  of  the  nine  key  performance  areas.  The  BARS 
instrument  matches  behavioral  indicators  elicited  as  part  of  the  Mastery  Model  development  with 
the  five  levels  of  learning  and  performance,  from  novice  to  expert.  As  a  result,  observable 
behaviors  related  to  squad  leader  tactical  thinking  can  be  rated  as  indicative  of  performance  at 
the  novice,  advanced  beginner,  proficient,  competent,  or  expert  level  of  mastery. 

Review  and  Selection  of  Available  Instruments 

To  support  the  development  of  the  initial  battery,  we  conducted  a  meta-review  of  research- 
supported  assessment  methods  for  the  remaining  16  constructs  (all  competencies  and  CARS,  and 
the  general  skill  of  decision  making)  and  made  recommendations  for  potential  “off-the-shelf’ 
instruments  to  include  in  the  battery  for  this  population.  The  meta-review  was  conducted 
simultaneously  with  model  development.  Initial  definitions  obtained  from  the  literature  were 
used  to  guide  both  this  review  of  the  literature  and  subsequent  development  of  operational 
definitions  that  resulted  from  model  development  as  noted  above.  The  definitions  obtained  from 
the  literature  and  a  description  of  the  instruments  retained  and  included  in  the  Initial  SUDM 
Assessment  Battery  can  be  found  in  Vogel-Walcutt,  Ross,  Smith,  &  Brown  (2012). 

A  meta-review  of  the  extant  literature  for  each  identified  construct  was  conducted.  Specifically,  a 
preliminary  investigation  of  internet  resources  at  large  (e.g.,  databases,  search  engines,  military 
reports)  was  conducted  yielding  25  possible  assessments  (Review  Round  I).  An  additional 
systematic  search  of  the  Google  Scholar®  database  was  conducted  (Review  Rounds  II  &  III). 
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For  this  search,  each  of  the  16  identified  constructs  was  paired  with  one  of  five  qualifiers  (scale, 
assessment,  inventory,  review,  and  meta- analysis)  yielding  80  combinations  of  search  terms.  For 
each  set  of  terms,  the  100  most  relevant  articles  published  between  1980  and  2012  were 
considered.  To  be  further  reviewed,  articles  had  to  note  either  a  specific  scale  that  assessed  the 
construct  or  be  identified  as  a  review  or  meta-analysis  of  several  articles  that  reviewed  the 
nature,  training,  or  assessment  of  the  skill.  In  the  next  phase,  each  article  was  reviewed  to 
determine  if  the  scale  noted  was  likely  to  be  (a)  military  relevant,  (b)  age  appropriate,  (c)  usable 
with  normal  populations  (e.g.,  not  for  patients  with  brain  traumas),  and  (d)  readily  available. 
Finally,  articles  were  required  to  supply  the  assessment  tool  in  the  article,  leaving  101  that  could 
be  easily  obtained.  Combining  Round  I  and  II  results,  126  total  articles  were  considered  for  scale 
review.  During  Round  III,  the  remaining  scales  were  obtained  and  assessed  using  the  same 
criteria  noted  above.  Fifteen  were  retained  from  this  round,  bringing  the  total  reviewed  to  141. 
Each  of  the  possible  assessment  tools  was  analyzed  to  determine  if  they  met  several  criteria: 
empirically  validated,  time  to  complete,  ready-to-use,  administrator  (self,  researcher,  and 
instructor),  military  focus,  training  specific  (is  the  training  program  expected  to  affect  scores  on 
this  assessment?),  and  quantitative/qualitative  (is  the  test  quantitative  or  qualitative  in  nature?).  A 
total  of  44  scales  were  retained  following  this  analysis. 

Once  this  final  set  of  acceptable  scales  was  identified,  it  was  necessary  to  reduce  the  number  of 
included  scales  to  increase  administration  efficiency  and  reduce  redundancy.  Therefore,  a  team 
of  scientists  reviewed  each  of  these  scales  in-depth  considering  primarily  their  relevance  to  the 
military  and  their  construct  validity  as  it  pertains  to  the  military’s  needs.  From  this  final  analysis, 
the  preliminary  set  of  scales  was  solidified.  Specifically,  scales  were  excluded  because  they 
lacked  military  relevance,  did  not  represent  the  construct  as  was  defined  by  our  review,  and/or 
were  redundant  to  other  scales  that  tested  the  same  construct.  Additionally,  several  scales  were 
transferred  to  a  different  construct  category  than  originally  classified  due  to  individual  scale 
items  better  matching  the  updated  construct  definitions. 

Twenty-seven  scales  were  retained  for  the  final  version  of  the  Initial  SUDM  Assessment  Battery 
(see  Figure  1).  Additional  review  and  possible  construction  of  sccdes  or  performance  tests  will  be 
needed  for  anomaly  detection,  and  change  detection.  Assessment  instruments  developed  in 
house  for  level  of  mastery,  overall  decision  making,  situational  assessment,  and  sensemaking  are 
also  shown  in  the  figure. 
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Measure  identified  or  constructed  Under  investigation 


Problemsolving 

•  Whimbey  Analytical  Skills 
Inventory  (WASI) 

•  Personal  Problem-Solving 
Inventory 

Perspect  ive  Taking 

•  Ambiguity  Tolerance  -  Cross 
Cultural  Scale 

•  Social  Awareness  Inventory 

Decision  Making 

•  Decision  Requirement  Interview 

•  Situational  Judgment  Test 

•  Melbourne  Decision  Making 
Questionnaire 

Sensemaking 

•  Situational  Judgment 

Test 

Ambiguity  Tolerance 

•  Dichotomous  Thinking 
Inventory 

•  Multiple  Stimulus  Types 
Ambiguity  Tolerance  (MSTAT) 

•  Measure  of  Ambiguity 
Tolerance  (MAT  50) 

Cognitive  Flexibility 

•  Cognitive  Flexibility  Scale 

Situational  Assessment 

•  Situational  Judgment  Test 

Level  of  Mastery 

•  Behaviorally  Anchored 
Rating  Scale 

Resilience 

•  Connor-Davidson  Resilience  Scale 

•  Brief  Resilience  Scale 

•  Brief  Resilient  Coping  Scale 

Metacognition 

•  State  Metacognitive 

Inventory 

•  Metacognitive  Awareness 
Inventory 

Analytical  Reasoning 

•  Metacognitive  Activities 
Inventory 

Adaptability 

•  Self  Report  Adaptive  Force  Scale 

•  Situational  Judgment  Test 

Adaptive  Force  Scale 

Change  Detection 

Anomaly  Detection 

Attentional  Control 

•  Neurocognitive  Assessment 

•  Action  Control  Scale 

•  Mindful  Attention  Awareness 
Scale 

Self-Regulation 

•  Difficulties  in  Emotion  Regulation 
Scale 

•  Problem  Solving  Scale 

Self-Awareness 

•  Freiburg  Mindfulness 
Inventory 

•  Situational  Self- 
Awareness  Scale 

©  Cognitive  Performance  Group  29Nov2012 


Figure  1.  Constructs  and  Initial  SUDM  Assessment  Battery  Instruments 

Usability  Testing  of  Off-the-Shelf  Instruments  and  BARS 

Two  rounds  of  usability  testing  (Phillips,  Vogel- Walcutt,  &  Lenhoff,  2012)  were  conducted.  The 
first  round  of  testing  was  conducted  in  October  2012.  Its  purpose  was  to  examine  the  ease  of  use 
and  time  required  for  completion  of  the  off-the-shelf  instruments  identified  as  candidates  for  the 
SUDM  Assessment  Battery  Pilot  Study.  The  second  round  of  testing  was  conducted  in 
November  2012.  The  purpose  of  the  second  round  of  usability  testing  was  to  examine  the  utility 
of  the  BARS  instrument,  specifically  the  goodness  of  the  BARS  scenario  and  its  ability  to 
capture  variability  in  performance. 

For  Round  One,  six  sergeants  from  Lima  Company,  3ld  Battalion,  25th  Marines,  based  in 
Columbus,  Ohio,  were  recruited  for  participation  by  TECOM.  All  participants  were  Infantry 
reservists  with  0311  (rifleman)  Military  Occupational  Specialties  (MOSs).  Each  had  served  in  a 
squad  leader  billet.  All  Round  One  participants  completed  all  the  off-the-shelf  tests  and  for  each 
test,  they  completed  a  short  survey.  For  Round  Two,  six  sergeants  from  the  School  of  Infantry- 
East  based  at  Camp  Lejeune,  North  Carolina,  were  identified  as  fitting  the  criteria  and  recruited 
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for  participation  by  the  Advanced  Infantry  Training  Battalion-East  (AITB-E).  All  participants 
were  instructors  at  SOI-E.  They  represented  0311,  0331  (machine  gunner),  and  0341 
(mortarman)  MOSs.  Each  had  served  in  a  squad  or  section  leader  billet.  All  Round  Two 
participants  were  interviewed  using  the  BARS  protocol  and  provided  scenario  feedback  during 
the  course  of  the  interview. 

A  subjective  assessment  of  the  usability  test  participant  feedback  suggests  two  of  the  off-the- 
shelf  instruments  tested  are  potentially  problematic  as  Assessment  Battery  candidates:  (1)  the 
Neurocognitive  Assessment  (Attentional  Control  construct)  and  (2)  the  SJT  Adaptive  Force 
Scale  (Adaptability  construct).  These  instruments  were  the  only  measures  judged  to  be  difficult 
to  understand  by  the  majority  of  the  three  participants  completing  them,  as  indicated  by  ratings 
related  to  instruction/question  clarity,  or  additional  comments  suggesting  confusion  or  trouble 
interpreting  the  response  choices.  Additional  feedback  will  be  collected  during  a  future  usability 
testing  session  for  the  SJT  Adaptive  Force  Scale.  The  Neurocognitive  Assessment  scale  will  be 
assessed  during  the  conduct  of  the  pilot  study.  The  results  of  these  additional  feedback  activities 
will  inform  a  future  decision  as  to  whether  to  drop  these  instruments. 

Each  BARS  Interview  session  lasted  between  35  and  45  minutes.  As  to  the  question  of  scenario 
realism  and  clarity,  participants  reported  the  scenario  to  be  sound  and  accurate;  no  revisions 
would  be  necessary  to  ensure  depiction  of  a  sufficient  mission  statement,  a  realistic  background 
situation,  or  a  relevant  tactical  situation.  As  to  the  question  of  whether  the  pause-point  queries 
effectively  elicited  decisions,  rationale,  priority,  concerns,  and  courses  of  action,  all  participants 
reported  the  ability  to  answer  the  questions  without  issue.  During  the  course  of  the  testing,  the 
interviewers  tested  out  the  addition  of  five  questions  at  the  final  pause  point,  and  found  those 
questions  to  be  valuable  for  eliciting  additional  information  as  to  participant  tactical  thinking. 

The  new  questions  were: 

•  What  orders  would  you  give  your  team  leaders? 

•  What  are  your  tactical  advantages? 

•  What  are  your  tactical  disadvantages? 

•  Is  there  additional  information  you  would  seek  at  this  time? 

•  What  do  you  believe  will  happen  next? 

Interviewers  found  the  question  “What  are  your  questions?”  to  fit  the  flow  of  the  questioning  in 
only  one  of  the  six  instances,  and  therefore  decided  to  eliminate  it  from  future  iterations. 

As  to  the  question  of  whether  the  scenario  was  likely  to  produce  variation  in  responses  as  a  result 
of  respondent  proficiency  level  differences,  participants  reported  the  scenario  to  be  too  simple. 
Several  resolutions  were  suggested,  and  the  following  will  be  implemented  prior  to 
commencement  of  the  pilot  study: 

•  Assign  the  respondent  to  be  the  main  effort  vice  the  supporting  effort. 

•  Include  more  civilians  in  the  streets  who  would  be  in  the  line  of  fire. 

•  Include  structures  of  varying  height,  to  prompt  additional  decisions  about  how  to  use  the 
terrain. 
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•  Do  not  color  the  structures  on  the  map  green,  indicating  to  participants  that  they  are 
cleared  and  secure,  which  is  antithetical  to  the  scenario  text. 

Method  for  Developing  Custom  Instruments 

The  Situational  Judgment  Test 

Background 

Situational  Judgment  Tests  (SJTs)  are  measurement  tools  that  are  typically  designed  to  assess 
knowledge,  skills,  cognitive  abilities,  values,  and  attitudes.  They  consist  of  job-related  scenarios 
that  describe  a  dilemma  requiring  individuals  to  apply  key  knowledge,  skills,  and  abilities 
(KSAs)  in  order  to  interpret,  judge,  and  respond  to  the  item.  Studies  using  meta-analysis 
techniques  have  shown  SJTs  to  have  superior  validity  over  traditional  performance  measurement 
techniques  (McDaniel,  Hartman,  &  Grubb,  2003;  McDaniel,  Morgeson,  Finnegan,  Campion,  & 
Braveman,  2001).  For  this  reason,  an  SJT  was  deemed  suitable  for  measuring  the  cognitive 
competencies  and  CARS  that  support  maneuver  squad  leader  performance  areas  and  decision 
making.  The  specific  constructs  measured  with  the  SJT  include  sensemaking,  situational 
assessment,  and  overall  decision  making;  no  standardized  measures  were  identified  as  part  of  the 
literature  review  for  these  three  constructs. 

Methodology 

Development  of  the  SJT  consisted  of  four  phases  with  Phases  3  and  4  still  in  progress.  Phase  1 
involved  converting  results  from  the  critical  incident  section  of  the  mastery  model  interviews  to 
scenarios.  Phase  2  required  in-house  SMEs  to  review  and  finalize  the  draft  scenarios  and 
responses.  Following  SME  review,  usability  testing  will  be  conducted  in  Phase  3.  In  Phase  4,  the 
response  format  for  each  item  will  be  determined. 

Phase  1:  Scenario  and  Response  Choice  Development 

Each  SJT  item  consists  of  a  brief  scenario  with  four  possible  courses  of  action.  The  scenarios  and 
courses  of  action  are  based  on  real  life  experiences  from  SMEs  interviewed  as  part  of  the 
mastery  model  development.  The  situations  were  developed  from  critical  incidents  revealed 
during  the  interviews.  Interviewees  were  asked  to  describe  situations  where  they  (or  someone 
else)  had  to  face  a  difficult  problem  and  ways  a  squad  leader  at  different  levels  of  proficiency 
would  respond.  The  incidents  were  used  as  a  framework  for  developing  scenarios  that  measure 
the  cognitive  competencies  and  CARS.  Scenarios  were  further  edited  to  ensure  each  item  was  of 
similar  length  and  format  (McDaniel  &  Nguyen,  2001).  A  readability  analysis  was  conducted  to 
verify  that  the  language  would  not  add  an  unnecessary  cognitive  load  on  the  examinee.  A  total  of 
25  items  were  developed  for  review. 

Phase  2:  SME  Review 

In  Phase  1,  scenarios  were  constructed  from  the  interviewees’  incident  reports,  but  changes  were 
made  to  the  events  to  ensure  the  cognitive  competencies  and  CARS  of  interest  were  being 
measured.  Due  to  these  changes  from  the  original  incidents,  it  was  necessary  to  undergo  a 
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preliminary  in-house  SME  review  prior  to  usability  testing.  The  purpose  of  the  in-house  SME 
review  was  to  ensure  that  modifications  to  the  scenarios  did  not  affect  the  realism  and  feasibility 
of  the  situations.  Two  SMEs  were  asked  to  review  all  scenarios  and  response  choices  and 
determine  whether  they  still  addressed  the  key  elements  of  the  original  story.  Feedback  from  the 
SMEs  was  evaluated  and  incorporated  to  finalize  the  25  SJT  items. 

Phase  3:  SJT  Usability  Testing 

Usability  testing  is  to  be  conducted  at  Camp  Lejeune,  North  Carolina,  with  six  SMEs  to 
determine  appropriateness,  face  validity,  and  clarity  of  the  items.  Each  participant  will  be 
provided  a  test  booklet  that  includes  SJT  items  on  separate  pages  with  instructions  and  space  for 
correcting  each  item.  For  each  item  the  SMEs  will  be  asked  to  answer  the  following  questions: 

1.  Does  the  scenario  portray  something  that  could  realistically  happen?  If  not,  what  change 
would  you  make? 

2.  Are  the  response  options  provided  realistic?  If  not,  what  would  you  change?  It  may  help 
to  consider  what  you  would  do  in  that  scenario. 

3.  Please  rank  the  response  options  from  best  to  worst  answers. 

Once  the  test  booklets  are  completed  and  returned,  the  responses  will  be  analyzed  and  final 
revisions  will  be  completed.  The  ranking  of  the  response  choices  will  be  used  to  develop  the 
response  format  and  scoring  technique. 

Phase  4:  Item  Response  Format 

After  the  completion  of  Phase  3,  the  item  response  format  will  be  developed.  Three  response 
formats  are  being  considered:  (1)  Multiple-choice,  which  requires  the  selection  of  an  action  or 
interpretation  as  appropriate/inappropriate  (Motowidlo,  Dunnette,  &  Carter,  1990);  (2)  Likert 
scales,  which  require  the  assessment  (e.g.,  the  effectiveness  of  each  action  or  interpretation)  of 
each  response  option  in  the  scenario  (Wagner  &  Stember,  1985;  Legree,  Heffner,  Psotka, 
Mesker,  &  Martin,  2003);  (3)  Open-ended  responses,  which  are  more  difficult  to  analyze  but 
allow  the  participants  to  freely  write-in  their  actions  or  interpretations  (Psotka,  Streeter, 
Landaver,  2004). 

Answers  scored  as  appropriate  will  be  based  on  SME  feedback  and  ranking  from  Phase  3. 
Scoring  guide  and  techniques  will  be  developed  once  the  response  format  has  been  finalized. 
Scoring  techniques  may  range  from  statistical  analysis  for  the  Likert  and  multiple-choice  options 
to  qualitative  analysis  for  the  open-ended  responses.  The  final  set  of  SJT  items  will  consist  of 
SME  validated  scenarios  and  items  developed  according  to  guidelines  in  the  scientific  literature. 
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The  Decision  Requirements  Interview 

Background 

To  more  comprehensively  address  the  lack  of  a  standardized  measure  for  addressing  the  overall 
decision  making  construct,  we  generated  a  new  instrument  called  a  Decision  Requirements 
Interview  (DRI). 

Decision  making  performance  is  difficult  to  measure  objectively,  especially  in  ill- structured, 
cognitively  complex  domains  such  as  small  unit  leadership.  These  domain  types  are 
characterized,  almost  by  definition,  as  allowing  for  multiple  decisions  that  will  satisfy  the 
requirement  given  a  real  world  dilemma  (e.g.,  Klein,  1998).  The  existence  of  many  correct 
answers  complicates  the  goal  of  assessing  performance  in  any  standard  manner.  In  the  past, 
researchers  have  had  some  success  using  SME  ratings  to  gauge  the  acceptability  of  an 
individual’s  course  of  action  (COA)  in  response  to  a  tactical  dilemma,  thereby  enabling  some 
degree  of  measurement  of  decision  performance.  However,  SMEs  are  not  always  readily 
available,  and  variability  exists  often  across  SME  judgments  of  quality  due  to  differences  in  their 
past  experiences  (e.g.,  McCloskey,  Pliske,  Klein,  Heaton,  &  Knight,  1997;  Pliske,  Militello, 
Phillips,  &  Battaglia,  2001). 

Attempts  to  focus  on  outcome  measures  to  judge  decision  performance,  such  as  mission 
accomplishment,  casualties,  or  number  of  kills,  are  likewise  problematic.  The  determination  of 
meaningful  outcome  measures  is  not  always  straightforward.  For  example,  a  desired  outcome 
may  be  to  reach  an  objective  or  collect  on  an  information  requirement  within  an  allotted  amount 
of  time.  However,  this  measure  does  not  account  for  the  manner  in  which  the  outcome  is 
achieved.  Often  a  number  of  process  measures  must  also  be  identified  to  take  into  account  the 
full  range  of  mission  goals,  such  as  building  collaborative  relationships  with  local  populations  or 
host  nation  forces  or  addressing  threats  as  they  become  apparent.  Furthermore,  achievement  of  a 
desired  outcome  is  sometimes  attributable  to  good  fortune  or  decisive  execution  of  a  mediocre 
decision,  vice  a  well-conceived  COA.  The  reverse  can  also  be  true:  a  strong  COA  can  fail  to 
meet  the  desired  outcomes  for  reasons  that  are  beyond  the  control  of  the  respondent.  For  these 
reasons,  measuring  decision  skill  based  solely  on  desired  outcomes  does  not  always  produce  a 
reliable  assessment. 

The  critical  factors  for  good  decision  making  include  an  individual’s  recognition  and 
interpretation  of  relevant  information  and  indicators  from  the  environment,  understanding  of  the 
complete  goal  set  pertinent  to  the  situation,  and  typical  ways  of  apply  tactical  and  technical  skills 
to  the  situation  to  solve  the  tactical  dilemma  (e.g.,  Klein,  1998).  Our  primary  objective  in 
developing  the  DRI  was  to  produce  a  technique  that  is  not  reliant  on  SME  ratings,  allows  for 
multiple  acceptable  COAs,  enables  differentiation  between  acceptable  and  superior  decision 
making,  and  bases  ratings  on  an  individual’s  thought  process  and  rationale  rather  than  the 
outcomes  of  his  performance.  To  do  so,  we  drew  on  the  practice  of  Cognitive  Task  Analysis 
(CIA). 

The  CTA  suite  of  methodologies  offers  a  means  of  identifying  and  describing  cognitive  tasks  and 
demands  associated  with  skilled  performance  in  a  domain  (Crandall,  Klein,  &  Hoffman,  2006). 
The  cognitive  performance  elements  are  often  captured  from  SMEs  to  model  decision  expertise 
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in  the  domain.  The  analysis  outcomes  can  be  used  to  design  expert  systems,  to  create  training,  or 
to  design  interfaces. 

A  decision  requirements  analysis  is  one  technique  for  identifying  and  documenting  the  decisions 
and  judgments  required  to  perform  a  task  or  set  of  tasks  within  a  domain,  along  with  their 
associated  cognitive  requirements.  It  typically  relies  on  critical  incident  data  and  decomposes  a 
data  set  consisting  of  several  incidents  into  the  decisions  and  cognitive  requirements  common 
across  the  incidents.  In  a  decision  requirements  analysis,  the  following  cognitive  elements  are 
often  delineated  for  each  decision  or  judgment: 

•  Perceptual  cues  that  are  sensed  -  seen,  heard,  smelled,  touched,  or  tasted  -  from  the 
immediate  environment  and  used  to  inform  the  decision. 

•  Background  factors  or  knowledge  held  prior  to  the  immediate  situation  and  considered 
when  making  the  decision.  Common  background  factors  for  small  unit  leaders  include 
the  mission  statement  and  commander’s  intent,  intelligence  about  the  enemy,  and 
knowledge  of  the  locals’  pattern  of  life. 

•  Reasons  why  the  decision  or  judgment  can  be  difficult. 

•  Strategies  or  aspects  of  an  individual’s  expertise  applied  to  the  decision. 

•  Goals  or  objectives  associated  with  the  decision,  or  what  the  decision  maker  is  really 
trying  to  achieve  by  making  the  decision. 

•  Errors  commonly  made  by  inexperienced  performers. 

The  outcome  of  a  decision  requirements  analysis  is  a  decision  requirements  table  (DRT)  in 
which  these  performance  elements  are  listed  for  every  decision  or  judgment. 

By  modifying  the  decision  requirements  analysis  technique  into  the  DRI,  we  are  creating  an 
elicitation  and  assessment  tool  that  will  similarly  deconstruct  the  thought  processes  and  rationale 
of  study  participants.  The  objective  is  to  produce  a  better  understanding  of  how  and  why 
individuals  make  certain  decisions  and  judgments  -  what  cues  and  factors  inform  decisions,  what 
they  are  ultimately  trying  to  achieve,  and  their  understanding  of  potential  performance  errors.  By 
capturing  and  examining  the  cognitive  processes  and  rationale  behind  the  decision,  we  can  better 
assess  the  strength  of  the  cognitive  performance. 

Methodology 

The  DRI  was  conceptualized  as  an  interview  technique  consisting  of  a  tactical  scenario  with  a 
series  of  events  requiring  respondents  to  make  sense  of  the  events  and  decide  upon  a  COA  for 
each  event.  The  administration  protocol  was  to  include  question  sets  to  elicit  the  respondents’ 
understanding  of  the  situation  and  considerations  for  decision  making,  and  a  DRT  to  record  those 
considerations  during  the  interview.  The  resultant  DRT  would  then  be  scored  against  a  master 
DRT;  points  would  be  awarded  for  noticing  and  correctly  interpreting  relevant  cues,  considering 
important  factors,  identifying  goals  to  be  achieved,  and  selecting  a  workable  COA.  Once  this 
interview  technique  is  modified  to  a  point  that  its  administration  can  be  standardized  and  it 
reliably  captures  decision  performance  data  as  desired,  our  goal  will  be  to  modify  its 
implementation  for  use  with  any  high-quality  tactical  scenario,  and  for  administration  by  non¬ 
researchers. 
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Phase  1:  Scenario  Selection  and  Modification 

The  DRI  development  began  with  the  selection  of  a  tactical  scenario.  Criteria  for  scenario 
selection  included  a  series  of  events  offering  multiple  opportunities  for  decision  making;  a  range 
of  possible  decisions;  moderate  to  high  complexity;  and  relevance  to  a  maneuver  squad  leader. 

Pre-existing  scenarios  were  identified  and  reviewed  to  select  a  scenario  best  matching  the 
criteria.  Two  primary  scenario  sets  were  considered:  a  set  of  12  Army  platoon  level  decision- 
centered  scenarios  set  in  a  Military  Operations  in  Urban  Terrain  (MOUT)  context  (Phillips  et  al., 
2001),  and  a  set  of  two  five-part  Marine  Corps  squad  level  interactive  decision-centered 
scenarios  created  for  employment  in  the  Infantry  Immersion  Trainer  (ITT)  at  Camp  Pendleton, 
California  (Ross,  Becker,  &  Lindsey,  2010).  Ultimately,  we  selected  the  Engagement  at  Jafarani 
scenario,  a  rifle  squad  combat  patrol  set  in  Afghanistan,  from  the  latter  set  of  IIT  scenarios. 

The  Engagement  at  Jafarani  scenario  was  modified  to  meet  the  DRI  objectives.  The  scenario 
was  originally  constructed  for  implementation  in  a  combined  live  and  virtual  training  facility, 
across  five  training  days  where  a  new  fragmentary  order  was  provided  at  the  start  of  each 
training  day.  We  selected  a  subset  of  the  scenario  events  to  include  in  the  DRI  scenario,  and  then 
reconstructed  it  into  a  paper- and-pencil  scenario  consisting  of  five  segments.  Events  were 
selected  with  the  goal  to  represent  a  range  of  operational  decision  types  and  performance  areas 
from  the  Maneuver  Squad  Leader  Mastery  Model.  Perceptual  cues  and  indicators  originally 
presented  via  live  or  virtual  scenario  events  were  re-written  for  introduction  in  text  and  images. 

A  mission  order  was  created  from  the  Engagement  at  Jafarani  mission  and  fragmentary  orders  to 
ensure  respondents  understood  the  mission,  the  operational  context,  friendly  and  enemy 
situations,  unit  tasks,  and  assets  available. 

Segment  1  of  the  scenario  is  receipt  of  the  patrol  order  and  requires  individuals  to  plan  and 
prepare  the  patrol  mission.  In  Segment  2,  participants  must  respond  to  a  team  leader  who  is  still 
shaken  up  from  a  previous  improvised  explosive  device  (IED)  event  and  a  fire  event  by  host 
nation  forces  on  locals  seen  in  the  fields  with  a  weapon.  The  third  Segment  takes  place  in  the 
commercial  sector  of  the  village  and  includes  indicators  of  suspicious  behavior  to  which  the 
individual  must  respond.  In  Segment  4,  the  participant  conducts  an  information  collection 
session  with  a  local  informant.  Finally,  in  Segment  5,  the  participant  must  respond  to  indicators 
of  a  possible  vehicle-borne  IED.  As  a  result  of  all  five  segments,  respondents  make  decisions 
related  to  the  following  maneuver  squad  leader  key  performance  areas:  Tactical  Skills/Tactical 
Thinking;  Adaptability/Flexibility;  Communication;  and  Job  Knowledge. 

Phase  2:  Administration  Protocol 

The  administration  protocol  consists  of  question  sets  at  each  of  the  five  segment  breaks  and  a 
DRT  framework  within  which  to  record  participant  responses  to  the  questions. 

We  began  by  constructing  the  DRT  framework  representing  the  response  data  to  be  collected. 

We  identified  the  following  typical  DRT  categories  of  data  as  pertinent  to  the  DRI  objectives: 

•  The  dilemma,  decision  point,  problem,  or  issue  requiring  a  decision. 
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•  Perceptual  cues  recognized  as  relevant  and  interpreted  to  support  effective  decision 
making. 

•  Background  factors  recognized  as  relevant  and  considered  when  making  the  decision. 

•  The  decision  made  or  COA  taken  by  the  participant  in  response  to  the  dilemma. 

•  Common  errors  associated  with  making  the  decision. 

•  Job  knowledge  applied  when  making  the  decision. 

These  six  categories  of  decision  requirements  became  the  column  headers  in  the  DRT 
framework.  Each  was  defined  in  the  framework,  and  examples  were  provided.  In  addition,  we 
constructed  a  completed,  or  master,  framework  to  identify  the  particular  dilemmas,  cues,  factors, 
decisions,  common  errors  and  job  knowledge  intended  for  participants  to  report,  as  well  as  the 
performance  area  represented.  The  master  DRT  framework  was  adapted  from  the  original 
Engagement  at  Jafarani  scenario  materials,  which  included  a  DRT  derived  from  CTA  interviews 
with  highly  skilled  squad  leaders. 

We  drafted  the  question  sets  at  each  segment  break  to  elicit  the  response  types  that  would 
populate  the  DRT  framework,  using  terminology  that  would  be  easily  understood  by  the  target 
population.  The  questions  were  sequenced  to  support  the  construction  of  the  DRT  during  the 
course  of  the  interview.  For  example,  participants  are  asked  to  first  identify  the  dilemmas  or 
problems  related  to  the  tactical  situation,  then  the  cues  and  factors  they  will  use  to  resolve  the 
dilemma  or  problem,  what  COA  they  will  pursue,  how  other  squad  leaders  might  make  mistakes 
in  handling  the  dilemma,  and  what  job  knowledge  they  used  to  resolve  the  dilemma. 

Phase  3:  Review,  Revision,  and  Usability  Testing 

An  in-house  SME  reviewed  the  scenario  and  modified  it  for  tactical  and  technical  accuracy, 
proper  terminology,  and  clarity  to  an  audience  of  Sergeants.  He  then  modified  the  master  DRT 
framework  to  reflect  his  scenario  revisions. 

The  DRI  will  be  subjected  to  usability  testing  with  six  Sergeants  representing  the  target 
audience.  The  protocol  will  be  administered  to  judge  the  clarity  of  the  scenario  and  question  sets, 
the  protocol’s  ability  to  elicit  the  desired  elements  of  cognitive  performance  and  decision 
considerations,  and  the  researchers’  ability  to  construct  the  DRT  during  the  course  of  an 
interview.  Following  usability  testing,  the  scenario,  question  sets,  and  DRT  framework  will  be 
modified  as  needed  to  increase  the  utility  and  ease  of  use  of  the  DRI  as  a  performance-based 
measure  of  decision  making. 

Results  and  Discussion 

A  version  of  the  Initial  SUDM  Assessment  Battery  (Ross,  Vogel- Walcutt,  &  Phillips,  2012)  was 
created  from  the  literature  review  and  the  custom  instruments  (BARS,  SJT,  and  DRI).  It  contains 
27  instruments  addressing  15  of  the  17  constructs  (excepting  anomaly  detection  and  change 
detection,  still  under  investigation).  This  version  is  subject  to  usability  testing  of  the  SJT  and  the 
DRI  to  be  carried  out  under  Option  1  of  the  contract.  Furthermore,  it  will  be  assessed  during  the 
conduct  of  a  pilot  study  and  modified  according  to  pilot  study  findings. 
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Following  usability  testing  of  the  SJT  and  DRI  instruments  and  instrument  revision,  the  pilot 
study  will  commence.  The  study  will  consist  of  five  administrations  of  the  battery  as  well  as 
completion  of  subjective  surveys  by  participants  and  their  supervisors.  The  administrations  will 
be  conducted  prior  to  the  cohort  of  students  entering  their  Advanced  Infantry  Courses,  prior  to 
entry  into  the  new  Infantry  Small  Unit  Leader  Course  (ISULC),  at  the  conclusion  of  ISULC,  6 
months  after  assignment  to  a  billet,  and  12  months  after  assignment.  The  purpose  of  the  data 
collection  is  to  pilot  test  the  battery.  Simultaneously,  battery  development  will  continue  as  off- 
the-shelf  measures  are  reduced  in  number,  custom  instruments  are  refined,  and  additional 
instruments  are  identified  or  developed  to  fill  gaps  or  better  address  constructs  than  do  initial 
instruments.  Some  changes  to  improve  efficiency  and  effectiveness  will  be  introduced  during  the 
pilot,  but  a  core  of  measures  will  be  maintained  across  the  pilot  administrations  to  insure 
consistent  repeated  measures  to  support  validity  of  insights  regarding  development  over  time. 
Some  changes  in  the  battery  resulting  from  analysis  of  the  pilot  data  will  be  introduced  only  after 
the  pilot  is  completed  and  the  SUDM  Assessment  Battery  is  finalized. 

Infantry  Sergeants  of  varying  levels  of  proficiency  will  complete  the  battery.  Data  gathered  will 
be  compared  against  hypothesized  results.  Instruments  will  be  examined  to  determine  if  the 
results  yield  data  that  allow  us  to  see  differences  in  the  group.  The  participant  group  is  currently 
expected  to  be  21  Marines.  Our  hypothesis  is  that  there  will  be  differences  within  the  group  at 
each  administration  that  allow  us  to  infer  differing  stages  of  development.  Across  time, 
instruments  that  are  measuring  constructs  we  hypothesize  should  change  due  to  training  and 
experience  will  be  examined  for  change  in  scores  over  time  (e.g.,  adaptability  and  level  of 
mastery).  Instruments  that  do  not  discriminate  among  the  participants  or  do  not  show  changes 
over  time  when  they  are  expected  to  do  so  will  be  considered  for  elimination.  We  expect  other 
instruments  related  to  constructs  that  may  be  considered  traits  not  to  show  changes  in  scores  over 
time.  Some  of  the  instruments  selected  were  designed  to  measure  traits.  However,  we  may  find 
that  all  of  the  constructs  are  amenable  to  change  due  to  training  and  experience,  and  we  must 
carefully  examine  the  results  to  determine  if  changes  are  present  and  show  trends  that  are  valid. 

Scales  that  conform  to  our  hypotheses  that  the  constructs  represent  traits  that  cannot  be  changed 
will  be  retained  only  if  we  believe  they  will  have  some  predictive  validity,  i.e.,  identify  potential 
good  decision-makers  in  future  psychometric  analysis  with  a  larger  group.  Scales  demonstrating 
high  correlations  with  other  scales  will  be  considered  for  elimination  on  the  basis  of  repetition  of 
the  same  underlying  construct  or  inability  to  add  value  to  the  battery.  In  these  cases,  it  is 
conceivable  that  whole  constructs  will  be  subsumed  under  others  or  eliminated  from  the  set  of  as 
a  result  of  these  indications  of  construct  similarity.  Significant  differences  will  be  explored 
further  to  determine  the  underlying  reasons  for  variations  from  expected  outcomes  to  insure 
instruments  conform  to  expectations  or  inform  new  conclusions  for  shaping  the  final  battery.  In 
addition,  all  instruments,  but  especially  the  DRI  and  BARS,  will  be  examined  for  continual 
refinement  to  increase  the  ease  of  administration,  and  revised  accordingly. 

Once  pilot  testing  is  underway,  we  expect  that  additional  gaps  may  emerge  in  the  battery.  For 
those  areas  where  the  currently  available  scales  are  too  cumbersome,  the  results  are  not  useful  for 
informing  training  practices,  or  where  no  scale  assesses  the  full  scope  of  a  construct,  new, 
military-targeted  scales  may  need  to  be  developed  and/or  measures  being  developed  under  other 
efforts  encompassing  the  constructs  must  be  identified  to  fill  the  gaps. 
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We  recommend  that  versions  of  the  actual  battery  be  restricted  to  official  use  by  the  research 
community  only.  Knowledge  of  the  items  which  can  be  the  subject  of  discussion  among  potential 
instructors  and  student  participants  in  the  subsequent  pilot  testing  can  invalidate  the  findings. 
Potential  use  of  the  scenarios  in  class  by  the  instructors  can  also  invalidate  findings.  Our  research 
scenarios  have  become  the  subject  of  instruction  in  previous  projects,  and  specific  requests  to 
retain  the  DRI  and  SJT  scenarios  for  instruction  have  been  made  already  during  the  course  of  this 
project,  making  this  a  realistic  threat  to  validity. 
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Appendix  A:  Operationalized  Definitions  and  Examples  of  SUDM 
Competencies  and  CARS 
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Competency 

Enabling  CARS 

Definition 

Example 

Sensemaking 

The  cognitive  process, 
driven  by  a  specific  goal, 
of  filtering  information  for 
relevancy  and  using  it  to 
construct  and  continually 
assess  an  explanation  of 
the  broad  or  specific 
situation,  often  in  the  form 
of  a  story,  in  order  to 
understand  how  and  why 
the  situation  evolved  and 
anticipate  what  might 
happen  next. 

While  patrolling  through  a  field  on  the  outskirts  of  a  village, 
the  squad  took  a  suspected  sniper  round.  The  maneuver 
squad  leader  judged  from  the  sound  of  the  shot  that  it  was 
a  .303  round.  He  and  the  rest  of  the  company  had  been 
tracking  a  highly  skilled  sniper  who  used  a  .303  and  had 
been  responsible  for  hitting  and  fatally  wounding  three 
Marines  from  the  company.  This  sniper  was  considered 
highly  skilled  because  he  had  deadly  aim  and  always  eluded 
capture.  Therefore,  the  sniper  was  considered  a  high  value 
target.  The  maneuver  squad  leader  considered  the  sound  of 
the  shot  to  narrow  down  the  sniper’s  position  to  a  general 
area.  Then,  he  conducted  a  perspective-taking  activity  to 
“flip  the  map”  and  visualize  the  terrain  from  the  sniper’s 
viewpoint  to  imagine  what  would  be  the  best  position  for  a 
sniper  attack  on  the  patrol.  Because  he  knew  this  sniper  was 
highly  skilled,  he  crafted  a  mental  story  that  the  man  would 
likely  be  positioned  in  the  best  possible  location,  one  that 
offered  concealment,  cover,  and  excellent  fields  of  fire. 

Based  on  his  sensemaking  and  subsequent  judgment  of  the 
sniper’s  location,  he  immediately  ordered  his  squad 
members  into  covered  positions  that  would  protect  them 
from  the  sniper,  but  also  orient  them  to  return  fire  and 
ultimately  kill  or  capture  the  sniper.  They  successfully 
neutralized  the  sniper. 

Perspective  Taking 

Visualizing  the  situation 
from  another’s  viewpoint 
and  assessing  his  or  her 
motivations  and 
objectives,  to  predict  his 

The  patrol  was  coming  up  upon  a  field  with  a  series  of 
compounds  in  the  distance.  The  patrol’s  route  had  them 
crossing  the  field  and  headed  toward  the  compounds.  As  he 
approached  the  field,  the  maneuver  squad  leader  considered 
enemy  TTP  for  the  region  and  looked  at  the  terrain  to 

A-2 


Competency 


Enabling  CARS 


Definition 


or  her  future  actions  and 
proactively  position  for  or 
take  advantageous  action. 


Critically  and  deliberately 
examining,  assessing,  and 
critiquing  one  or  more 
alternatives  or 
assumptions  in  the  context 
of  specified  goals  (e.g., 
the  mission)  and  against  a 
set  of  evaluative  criteria 
(e.g.,  intent,  timing, 
resources,  or  ROE). 
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Example 


identify  the  two  or  three  potential  positions  from  which  the 
enemy  may  attack  as  the  patrol  crossed  the  field.  This 
perspective  taking  activity  supported  decision  making  in 
that  the  maneuver  squad  leader  was  attuned  to  watching 
those  positions  more  closely  than  others,  and  began  to 
identify  courses  of  action  should  the  enemy  actually  attack 
from  those  positions. 

The  first  maneuver  squad  leader  in  the  platoon  typically 
implemented  a  much  greater  dispersion  between  his  fire 
teams  when  they  moved  in  formation  than  was  unit  SOP.  He 
used  an  analysis  of  the  mission,  terrain,  enemy  tactics,  and 
squad  capabilities  to  identify  that  a  greater  dispersion  would 
make  the  squad  more  effective  against  the  enemy.  His 
mission  had  his  squad  conducting  dismounted  patrols 
through  the  open  terrain  of  rural  Afghanistan.  The  enemy 
tactics  were  to  attack  from  a  distance  outside  the  range  of 
rifle  squad  weapons,  and  then  run  away  to  avoid  becoming 
decisively  engaged  with  the  Marines.  The  maneuver  squad 
leader  himself  was  senior  and  experienced,  meaning  he  was 
capable  of  maintaining  good  command  and  control  of  his 
squad  and  coordinating  effectively  with  his  weapons  teams 
to  direct  supporting  fires.  His  fire  team  leaders  were 
likewise  strong  and  able  to  function  autonomously  with 
general  tasking  and  intent  from  their  leader.  He  reasoned 
that  greater  dispersion  of  the  squad  would  enable  better 
coverage  and  response  to  enemy  attacks,  without  sacrificing 
the  ability  of  the  elements  to  be  mutually  supportive,  even  at 
those  distances. 


Competency 


Enabling  CARS 


Definition 


Realizing  through 
perceptual-cognitive 
processes  that  the 
presence  or  absence  of 
elements  or  patterns  of 
elements  in  the 
environment  is  off  the 
baseline  for  that  setting, 
and  therefore  requires 
more  explicit  reasoning  to 
locate  the  source  of  the 
anomaly  and  understand 
its  implications. 


Attending  to  relevant 
aspects  of  the  environment 
in  order  to  perceive  a 
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Example 


A  squad  took  contact,  and  one  of  the  fire  teams  became 
engaged  with  an  unknown  enemy.  The  fire  team  leader 
reported  to  the  maneuver  squad  leader  that  they  were 
engaged  by  enemy  fighters  who  looked  to  be  adult  men. 
However,  the  engagement  distance  was  far,  so  the  fire  team 
leader  did  not  get  a  good  look  at  the  combatants.  Soon  after 
the  squad  broke  contact,  the  maneuver  squad  leader  and  a 
team  of  Marines  patrolled  through  a  nearby  village.  As  they 
rounded  a  corner  in  the  village,  they  observed  three  young 
boys,  approximately  12-16  years  old,  look  at  them  then 
squat  down  to  begin  playing  a  dice  game.  The  maneuver 
squad  leader  judged  this  behavior  as  an  anomaly,  because 
the  boys  didn’t  begin  the  game  until  they  saw  the  Marines. 
He  then  engaged  in  a  sensemaking  activity  to  make  sense  of 
the  anomaly.  He  crafted  a  mental  story  that  these  boys  were 
following  the  enemy’s  common  tactic  of  attacking  Marines, 
and  then  attempting  to  blend  in  with  the  population.  He 
reasoned  that  brought  dice  with  them  as  part  of  their  plan  to 
appear  normal  and  innocent.  However,  being  young  adults, 
they  didn't  realize  that  their  sudden  change  of  activity  was 
an  anomaly  that  drew  suspicion.  While  the  fire  team  leader 
reported  that  the  enemy  that  engaged  them  were  older 
adults,  the  maneuver  squad  leader  could  easily  imagine  a 
story  where  these  three  boys  were  responsible  and 
attempting  to  cover  their  involvement.  He  immediately 
checked  the  boys  for  gunshot  residue,  and  all  three  tested 
positive.  He  detained  them. 

The  squad  was  toward  the  end  of  a  lengthy  patrol.  Upon 
retrograde  back  toward  base,  the  maneuver  squad  leader 
recognized  a  change  in  the  demeanor  of  the  villagers  they 


Competency 


Enabling  CARS 


Definition 


difference  in  one  or  more 
elements  in  the  situation, 
and  interpreting  that 
difference  to  support  one’s 
situational  awareness, 
understanding  of  baseline, 
or  immediate  threat 
assessment. 


Analytically  or  intuitively 
identifying  and  collecting 
information  from  multiple 
available  sources, 
including  one’s  own 
knowledge,  to  analyze 
relevant  factors  of  METT- 
TC  and  construct  an 
understanding  of  the 
situation  to  support  a 
specific  task  or  goal. 
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Example 


had  recently  passed  on  their  way  out.  The  villagers  were  still 
milling  about  as  they  had  been  previously,  but  they  had  a 
tenseness  about  them  that  was  different  from  the  first  time 
the  patrol  encountered  them.  The  tenseness  included  them 
paying  more  attention  to  the  Marines  than  they  normally 
would;  they  were  more  focused  on  the  Marines  than  usual. 
Typically  the  villagers  would  take  note  of  the  Marines  but 
then  go  back  to  their  business.  In  this  case,  they  continued  to 
keep  an  eye  on  the  Marines,  as  if  they  were  waiting  to  see 
the  action  that  would  ensue,  with  the  Marines  at  the  center 
of  it.  The  maneuver  squad  leader  interpreted  this  tenseness 
as  an  impending  attack.  He  knew  that  the  villagers  were  too 
intimidated  by  the  insurgents  to  initiate  a  talk  with  or  warn 
the  Marines.  He  suspected  they  were  hanging  around  to 
watch  the  attack  they  knew  was  coming.  And  his  suspicion 
was  correct;  the  patrol  came  under  fire  from  the  far  side  of 
the  village. 

As  part  of  a  company-sized  operation  in  Fallujah,  a  squad 
was  moving  in  formation  through  the  hostile  city  with  a 
sister  squad  to  one  flank,  but  no  friendlies  on  the  other 
flank.  The  platoon  was  beginning  to  encounter  resistance  in 
the  form  of  small  arms  fire.  One  Marine  had  fallen  out 
because  he  was  shot,  and  was  now  lying  in  the  middle  of  the 
street  to  the  squad’s  rear.  The  maneuver  squad  leader,  who 
was  nearest  to  him,  went  back  to  retrieve  him  while  the  rest 
of  the  patrol  halted  in  covered  positions.  As  the  maneuver 
squad  leader  began  talking  to  the  injured  Marine  to  assess 
his  injury,  the  Marine  held  his  index  finger  to  his  lips  as  a 
signal  to  “shhh,  be  quiet.”  The  maneuver  squad  leader 
listened  and  heard,  from  the  other  side  of  the  courtyard  wall 
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Competency 


Enabling  CARS 


Definition 


Example 


2  meters  to  his  left,  the  sound  of  guys  changing  magazines. 
He  now  knew  the  enemy’s  position,  and  he  had  a  sense  of 
the  brief  window  of  time  available  to  act.  He  knew  he  had  to 
immediately  pull  the  much  larger  Marine  to  safety  before 
the  hostiles  re-engaged.  That  meant  he  would  have  to  do  it 
himself  rather  than  his  first  instinct,  which  was  to  call  for 
other  Marines  to  help  carry  the  wounded  Marine  out  of  the 
street.  He  dragged  the  Marine  to  a  covered  position,  but  as 
he  was  doing  so,  he  himself  took  some  shrapnel  from  a 
grenade  underneath  his  flak  jacket,  on  the  shoulder.  Because 
he  was  in  pain,  he  forced  the  Marine  to  help  him  by  scooting 
himself  with  his  good  leg  while  the  maneuver  squad  leader 
pulled  him  to  safety. 


Adaptability 


Fluidly  modifying  or 
changing  one’s  planned 
actions  when  the  situation 
has  changed  from  what 
was  expected,  or  when  the 
typical  approach  or  plan  is 
rendered  less  effective 
than  necessary. 


A  prisoner  was  inadvertently  released,  and  a  maneuver 
squad  leader  was  given  the  task  to  capture  and  re-detain  the 
man,  and  bring  him  back  to  the  detention  facility.  The 
maneuver  squad  leader  had  detained  several  prisoners  in  the 
past  and  was  familiar  with  how  to  snatch  a  wanted  man 
from  a  residence,  flex-cuff  him,  and  load  him  into  a  vehicle 
to  transport  him  to  the  FOB.  However,  as  the  squad 
approached  the  man’s  residence,  it  became  apparent  that  the 
village  was  throwing  a  huge  party  to  celebrate  the  man’s 
release.  Over  a  hundred  friends  and  family  members  were  in 
or  around  the  residence,  celebrating  with  the  man.  The 
maneuver  squad  leader  quickly  assessed  that  a  hostile 
detention  of  the  man  would  backfire  on  him — his  squad  was 
vastly  outnumbered  which  would  encourage  the  villagers  to 
revolt,  and  he  and  his  Marines  would  be  forced  to  apply 
deadly  force  to  protect  themselves.  Instead  of  using  the 
typical  and  planned  approach  to  detaining  a  man,  the 
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Competency 


Enabling  CARS 


Definition 


Situational  Assessment  Analytically  or  intuitively 

identifying  and  collecting 
information  from  multiple 
available  sources, 
including  one’s  own 
knowledge,  to  analyze 
relevant  factors  of  METT- 
TC  and  construct  an 
understanding  of  the 
situation  to  support  a 
specific  task  or  goal. 
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MANEUVER  SQUAD  LEADER  instead  came  up  with  a 
non-hostile  ruse  to  get  him  into  his  custody.  He  told  the  man 
that  the  paperwork  for  his  release  had  been  improperly 
completed,  and  his  signature  was  required  back  at  the 
prison.  He  requested  politely  and  apologetically  that  the 
man  come  with  him  to  sort  out  the  mess,  and  then  he  would 
return  him  to  his  home.  The  man  put  up  no  resistance  and 
gladly  went  with  the  squad  to  further  cement  his  release. 

The  maneuver  squad  leader’s  adaptability  prevented  a 
skirmish  and  what  would  have  been  harmful  second  and 
third  order  effects. 


Two  squads  were  given  a  mission  to  conduct  a  raid  on  a 
compound  housing  known  opium  distributors.  The  squads 
were  to  be  helo-dropped  into  the  middle  of  the  village  to 
conduct  the  raid  mission.  The  maneuver  squad  leaders 
judged  several  challenges  associated  with  the  mission,  but 
nevertheless  worked  with  their  platoon  commander  and 
platoon  sergeant  during  planning  to  analyze  the  mission 
goals,  the  terrain  of  the  village  and  surrounding  the  landing 
zone,  the  enemy  capabilities  and  expected  resistance,  the 
civilian  population  in  the  village,  and  the  friendly  resources 
required.  A  plan  was  developed  as  a  result  of  the  mission 
analysis,  which  spanned  approximately  48  hours.  However, 
during  the  helo-transit  to  the  drop  site,  the  company 
commander  called  to  say  that  the  landing  zone  had  been 
changed  to  a  location  2  km  north  of  the  previously  planned 
drop  site.  The  squads  had  15  minutes  before  they  would 
touch  down,  and  therefore  only  15  minutes  to  reassess  the 
situation  and  re-plan  the  sticky  operation.  The  maneuver 
squad  leaders  had  a  solid  understanding  of  the  company  and 


Competency 


Enabling  CARS 


Definition 


Applying  knowledge  and 
principles  of  tactics  and 
leadership  differentially 
based  upon  the  unique 
demands  of  the  situation. 
Applying  knowledge 
learned  in  one  context  to 
multiple  relevant  contexts 


The  ability  to  calmly 
withstand  and  operate 
within  uncertain 
environments  by  delaying 
drawing  a  conclusion  or 
making  a  decision,  or  by 


A-8 


CPG-0001 AE-21  Decl  2 


Example 


platoon  commanders’  intent,  and  were  able  to  adapt  their 
planned  actions  to  reach  the  objective  based  on  a  re¬ 
assessment  of  the  terrain  and  enemy  they  would  encounter 
as  a  result  of  the  changed  landing  zone. 

A  dismounted  squad  was  moving  through  an  Iraqi  city  as 
part  of  a  platoon-sized  operation.  It  was  a  dirty  little  town, 
with  concrete  buildings,  dirt  roads,  ruts  and  junk 
everywhere.  They  expected  to  encounter  resistance.  As  they 
were  traveling  down  an  alleyway  with  the  squad  spread  out, 
the  maneuver  squad  leader  recalled  a  lesson  from  maneuver 
squad  leader  school:  the  long  axis  of  the  kill  zone  coincides 
with  the  long  axis  of  the  target.  He  calculated  that  if  they 
were  to  come  upon  an  enemy  machine  gun  position,  it 
would  be  oriented  down  the  alleyway.  In  the  current 
formation,  they  were  exposed  and  would  be  easily  picked 
off.  He  immediately  ordered  the  squad  of  ten  Marines  to 
split  in  two  sections  and  travel  down  parallel  roads  to 
provide  mutual  support.  He  reasoned  that  splitting  the  squad 
would  increase  its  survivability.  As  it  turned  out,  the  enemy 
did  in  fact  have  a  machine  gun  positioned  at  the  end  of  the 
alleyway,  and  by  splitting  the  squad  they  were  able  to  avoid 
being  trapped  and  more  effectively  engage  the  enemy 
position. 

The  squad  was  tasked  to  form  a  blocking  position  for 
another  squad’s  operation.  While  fulfilling  this  mission,  the 
squad  lost  the  antenna  for  the  electronic  counter  measures 
while  in  transit  and  went  back  to  an  open  field  to  look  for 
and  retrieve  it.  While  searching  in  the  field,  they  came  under 
fire.  They  had  no  communications  to  call  for  help  or  even 
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Competency 


Enabling  CARS 


Definition 


Example 


making  assessments  and 
decisions  in  the  face  of 
uncertainty. 


report  their  situation,  and  they  were  vulnerable  in  the  wide 
open  terrain.  The  maneuver  squad  leader  did  not  know  how 
many  enemies  were  involved.  He  didn’t  want  to  move 
forward  to  engage  them  and  kill  them  because  he  didn’t 
have  the  ability  to  call  in  a  casualty  evacuation  in  case 
someone  became  injured.  He  calmly  handled  the  situation. 
He  directed  his  fire  teams  to  bound  back  two  teams  at  a 
time,  with  the  third  team  suppressing  the  enemy  while  the 
others  moved.  Then  he  sent  one  of  his  fire  team  leaders  with 
the  backup,  short  distance  radio  to  get  close  enough  to  the 
other  squad  to  radio  them  for  assistance.  Eventually  they 
were  able  to  move  into  a  wadi  for  cover,  which  expedited 
their  egress. 


Attentional 

Control 


Activities  related  to 
maintaining  a  focus  on 
mission  completion 
despite  distractors 
including  stress,  boredom, 
fatigue,  and  emotion. 


The  Battalion  Gunner,  with  his  crew,  was  visiting  the  FOB 
in  Ramadi.  After  the  visit,  on  their  way  out  of  the  FOB,  they 
had  to  cut  across  a  float  bridge.  This  time,  the  vehicles  hit  a 
pressure  plate  IED,  and  they  lost  comms  with  everybody 
except  one  maneuver  squad  leader,  a  Sergeant.  The  Sergeant 
became  responsible  for  directing  all  the  traffic  to  respond 
and  help  the  Gunner’s  convoy,  and  communicating 
information  as  the  middleman  between  the  S3,  the  Battalion 
Commander,  and  the  Gunner.  This  was  a  massive 
communication  and  coordination  piece  for  the  maneuver 
squad  leader,  and  he  was  stressed  out  about  it.  He 
coordinated  a  ground  medevac  for  them.  He  knew  where  the 
Gunner  was  located  from  his  first  comm  with  him,  and  he 
could  see  them  on  the  G-Boss.  Instead  of  waiting  for  a  9- 
line  from  the  Gunner’s  vehicles,  he  immediately  launched  4 
gun  trucks  from  the  FOB  as  the  medevac.  He  prioritized 
that,  since  he  knew  where  they  were  and  knew  what 
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assistance  they  needed,  he  could  use  gun  trucks  and  get  the 
medevac  spun  up  immediately.  The  9-line  could  be  sent 
later.  The  medevac  reached  the  Gunner  within  5  minutes. 
The  maneuver  squad  leader  also  called  in  two  supporting 
units  to  come  in  and  cordon  off  the  area,  and  directed  their 
strategic  positioning  as  blocking  positions  based  on  his 
knowledge  of  the  terrain  right  outside  the  FOB.  In  all,  he 
managed  and  juggled  5  radio  nets.  “Because  I  was  under 
stress,  I  was  just  making  really  good  decisions.”  He 
managed  the  flood  of  information,  made  the  right  decisions 
as  to  where  to  set  up  the  blocking  positions,  and  was  able  to 
communicate  situation  updates  back  and  forth  between  the 
Gunner  and  the  S3  and  Battalion  Commander.  After  the 
event,  “. .  .my  company  commander  said  he  couldn’t  have 
done  things  that  day  that  I  did.  He  was  like,  “Where  the  hell 
did  that  come  from?”  I  was  like,  ‘Dude,  I  don’t  know,  I  was 
stressed  out  and  I  was  just  in  the  zone.’” 


A  maneuver  squad  leader  was  injured  in  a  firefight  in  a  city 
in  Iraq.  Because  no  vehicle  could  fit  through  the  alleys,  the 
casualty  evacuation  took  the  form  of  dismounted 
reinforcement  Marines  and  stretchers  for  those  who  needed 
them.  The  reinforced  unit  would  then  bound  back  and  out  of 
the  city  to  safety.  The  maneuver  squad  leader’s  injuries  were 
bad  enough  that  a  stretcher  was  called  for.  However,  he 
knew  that  if  he  got  on  a  stretcher,  it  would  require  two 
Marines  to  carry  him,  and  that  would  take  two  Marines  out 
of  the  fight  on  the  way  out  of  the  city.  To  set  an  example  of 
mental  toughness,  and  to  maximize  the  employment  of  his 
Marines,  he  refused  to  be  carried  on  a  stretcher  and  instead 
walked  on  his  own  as  the  unit  moved  out  of  the  city.  This 
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Self-Regulation 


Metacognition 


Monitoring,  assessing,  and 
adjusting  one's  own 
behavior  and  its  effects  in 
order  to  impact  the 
situation  in  a  way  that 
supports  mission,  unit,  or 
training  goals. 


maneuver  squad  leader  continued  to  lead  his  squad  through 
the  firefight  even  after  sustaining  the  injury. 

A  platoon  sergeant  was  involved  in  a  major  firefight  in 
Fallujah,  where  he  was  operating  with  one  of  the  squads. 
During  the  operation  he  was  hit  with  shrapnel  from  a 
grenade  blast  and  experienced  substantial  bleeding.  While 
the  corpsman  was  treating  him  in  the  middle  of  the  city,  he 
noticed  two  of  the  younger  Marines  watching  him  with  wide 
eyes,  their  faces  growing  white.  He  assessed  that  they  were 
terrified  that  their  platoon  sergeant,  who  is  the  most 
experienced  and  combat- wise  member  of  the  unit,  was 
seriously  injured  and  potentially  combat  ineffective.  He 
therefore  adjusted  his  behavior  by  making  light  of  the 
situation:  “Hey  corpsman,  did  the  shrapnel  mess  up  any  of 
my  tattoos?  I  hope  not  -  they  were  expensive!”  The  Marines 
immediately  began  laughing  at  his  horribly  misaligned 
priorities,  and  gained  confidence  that  “Staff  Sergeant  must 
not  be  hurt  too  badly  if  he’s  worried  about  his  tattoos!” 

After  lightening  the  mood  with  his  humor,  he  proceeded  to 
give  each  Marine  a  very  specific  task,  with  clear  direction, 
that  would  direct  their  attention  to  a  small  set  of  goals  and 
allow  them  to  feel  they  had  a  purpose  as  contributors  to  the 
fight.  He  regulated  his  behavior  to  use  humor,  despite  the 
pain  and  stress,  to  keep  the  two  Marines  from  shutting  down 
from  fear  in  the  middle  of  the  firefight. 


Activities  related  to 
considering  one’s  own 
thought  processes, 
including  assessments  of 


Marines  who  are  newly  billeted  as  maneuver  squad  leaders 
go  through  a  series  of  realizations  about  what  it  takes  to  be  a 
maneuver  squad  leader,  and  how  they  will  need  to  adjust 
their  thought  processes  and  behaviors  to  do  the  job.  One  of 
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Self-Awareness 


strengths  and  limitations 
or  developmental  needs,  in 
support  of  performing  or 
learning  the  job. 


the  early  realizations,  especially  challenging  for  Marines 
who  are  promoted  to  the  billet  within  their  current  platoon, 
is  that  they  must  think  and  act  like  a  leader  instead  of  a  peer. 
They  can  no  longer  be  drinking  buddies  with  their  Marines. 
They  can  no  longer  go  home  on  the  weekends  and  hang  out 
with  their  high  school  friends.  They  must  separate 
themselves  so  that  they  can  effectively  manage  the  welfare 
of  the  squad.  Another  realization  is  that  they  must  shift  from 
focusing  on  themselves  and  their  own  performance,  to 
focusing  on  their  Marines  and  the  squad’s  performance. 
Eventually  another  shift  of  focus  occurs,  from  focusing  on 
the  squad  actions  in  combat  to  focusing  on  the  enemy’s 
activities,  in  order  to  anticipate  and  stay  a  step  ahead  of  the 
enemy  at  all  times. 


Conscious  knowledge  of 
one’s  own  character, 
motives,  knowledge  base, 
and  skill  set  in  order  to 
request  information  or 
assistance  when  the 
requirements  of  the 
situation  call  for 
capabilities  beyond  one’s 
current  abilities. 


A  Sergeant  pulled  from  Security  Forces  was  billeted  as  a 
rifle  maneuver  squad  leader.  He  quickly  came  to  realize  that 
his  tactical  and  technical  proficiency  was  not  on  par  with  the 
other  maneuver  squad  leaders  in  the  platoon.  He  also 
realized  that  as  a  maneuver  squad  leader,  he  would  be 
expected  to  train  his  Marines  on  weapons,  equipment,  and 
tactics.  To  gain  their  respect  and  trust,  he  would  need  to  be 
more  knowledgeable  than  they.  So,  he  made  concerted 
efforts  to  study  manuals  and  tactical  pubs,  pull  information 
and  experience-based  knowledge  from  his  platoon  sergeant 
and  trusted  peers,  and  learn  all  he  could  from  every  training 
experience  he  encountered. 


Problem  Solving 


Identification,  definition, 
examination, 
prioritization,  and 


A  dismounted  squad  was  in  a  firefight  in  Fallujah.  They’d 
taken  three  casualties,  and  had  become  holed  up  in  an  Iraqi 
residence  they  turned  into  their  casualty  collection  point. 
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resolution  of  situations 
that  impede  task  or 
mission  accomplishment. 


They  had  captured  three  insurgent  fighters  flex-cuffed  in 
one  room  under  security  and  women  and  children  cordoned 
off  in  a  separate  room  at  the  back  of  the  house,  also  under 
security.  They’d  split  the  squad,  with  a  team  on  the  roof  in 
an  overwatch  position  pulling  360  degree  security,  and 
another  team  on  the  ground  floor.  There  was  fighting 
outside  all  around  them.  They  had  lost  comms  because  of 
the  structures  in  the  city  and  the  dense  urban  jungle.  A 
couple  members  of  a  CAAT  team  had  come  to  their  aid, 
with  a  corpsman  in  tow,  when  they  heard  the  shots  break 
out.  The  maneuver  squad  leader  knew  they  had  a  problem  - 
they  needed  to  get  out  and  get  help  quickly,  get  care  for 
their  wounded,  and  ensure  they  wouldn’t  become 
overwhelmed  by  the  enemy,  who  had  much  greater 
numbers.  Even  though  he  was  outranked  by  his  platoon 
sergeant  who  was  also  with  the  squad,  he  took  charge  and 
came  up  with  a  plan.  He  took  out  his  GPS,  stuck  it  in  his 
platoon  sergeant’s  face,  and  said,  “Hey,  I  know  where  we 
are,  I’m  going  to  go  get  some  help.”  He  continued,  saying 
the  Marines  from  the  CAAT  team  knew  the  position  of  their 
vehicles,  and  he  could  bound  back  to  the  vehicles  with  them 
and  call  for  help  from  there.  In  this  situation,  it  was  a 
brilliantly  reasoned  solution  to  the  problem.  Once  security 
was  posted,  the  platoon  sergeant  gave  the  order  to  execute. 
The  plan  worked,  and  the  squad  was  able  to  be  extracted 
successfully  from  their  position. 


Analytical  Reasoning 


Critically  and  deliberating 
examining,  assessing,  and 
critiquing  one  or  more 
alternatives  or 


A  squad  was  tasked  with  an  operation  to  detain  an 
individual  in  downtown  Ramadi.  The  platoon  commander, 
platoon  sergeant,  and  maneuver  squad  leader  all  believed 
they  knew  where  the  man  was  located,  based  on  the 
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I  as  sumptions  in  the  context 

of  specified  goals  (e.g., 
the  mission)  and  against  a 
set  of  evaluative  criteria 
(e.g.,  intent,  timing, 
resources,  or  ROE). 
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actionable  intelligence  they’d  received.  But,  when  the  squad 
went  into  the  house  and  checked  the  residents’  ID  cards, 
they  quickly  realized  they  were  in  the  wrong  house.  The 
maneuver  squad  leader  was  immediately  upset  with  himself 
for  thinking  to  plan  for  several  other  contingencies,  but  not 
the  simple  contingency  of  what  to  do  if  he  ended  up  at  the 
wrong  house.  During  the  planning  sessions,  since  he  had 
external  agencies  working  with  him,  he  had  focused  his 
attention  on  what  he  wanted  to  task  them  to  do,  what  he 
wanted  his  squad’s  security  posture  to  be,  and  so  forth.  He 
didn’t  think  about  the  “what  if  it’s  the  wrong  house?”  He 
didn’t  have,  as  he  called  it,  a  brush-off  plan.  Once  he 
realized  he  was  in  the  wrong  house,  he  knew  he  had  a 
problem  he  needed  to  resolve  -  smoothing  over  the  situation 
with  the  family.  The  squad  hadn’t  destroyed  anything  in  the 
house.  The  maneuver  squad  leader  was  civil,  apologized  to 
them,  and  asked  if  they  needed  any  water.  Then  he  gave 
them  a  case  of  water  off  one  of  the  Humvees.  That  became 
the  improvised  brush-off  plan.  Next,  he  had  to  analyze  what 
he  knew  about  the  target  from  the  planning.  He  reasoned 
that  they  knew  they  were  in  the  vicinity  of  the  target,  so  he 
had  to  be  in  one  of  the  residences  nearby.  He  made  the 
decision  to  go  to  the  house  on  the  right,  and  then  the  house 
on  the  left,  and  proceed  out  from  there  until  they  found  him. 
As  it  turned  out,  the  wanted  man  was  in  the  house  on  the  left 
-  the  third  residence  they  entered. 


